Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process

Gradient estimation in dendritic reinforcement learning

We study synaptic plasticity in a complex neuronal cell model where NMDA-spikes can arise in certain dendritic zones. In the context of reinforcement learning, two kinds of plasticity rules are derived, zone reinforcement (ZR) and cell reinforcement (CR), which both optimize the expected reward by stochastic gradient ascent. For ZR, the synaptic plasticity response to the external reward signal is modulated exclusively by quantities which are local to the NMDA-spike initiation zone in which the synapse is situated. CR, in addition, uses nonlocal feedback from the soma of the cell, provided by mechanisms such as the backpropagating action potential. Simulation results show that, compared to ZR, the use of nonlocal feedback in CR can drastically enhance learning performance. We suggest that the availability of nonlocal feedback for learning is a key advantage of complex neurons over networks of simple point neurons, which have previously been found to be largely equivalent with regard to computational capability

Models of Neocortical Layer 5b Pyramidal Cells Capturing a Wide Range of Dendritic and Perisomatic Active Properties

The thick-tufted layer 5b pyramidal cell extends its dendritic tree to all six layers of the mammalian neocortex and serves as a major building block for the cortical column. L5b pyramidal cells have been the subject of extensive experimental and modeling studies, yet conductance-based models of these cells that faithfully reproduce both their perisomatic Na+-spiking behavior as well as key dendritic active properties, including Ca2+ spikes and back-propagating action potentials, are still lacking. Based on a large body of experimental recordings from both the soma and dendrites of L5b pyramidal cells in adult rats, we characterized key features of the somatic and dendritic firing and quantified their statistics. We used these features to constrain the density of a set of ion channels over the soma and dendritic surface via multi-objective optimization with an evolutionary algorithm, thus generating a set of detailed conductance-based models that faithfully replicate the back-propagating action potential activated Ca2+ spike firing and the perisomatic firing response to current steps, as well as the experimental variability of the properties. Furthermore, we show a useful way to analyze model parameters with our sets of models, which enabled us to identify some of the mechanisms responsible for the dynamic properties of L5b pyramidal cells as well as mechanisms that are sensitive to morphological changes. This automated framework can be used to develop a database of faithful models for other neuron types. The models we present provide several experimentally-testable predictions and can serve as a powerful tool for theoretical investigations of the contribution of single-cell dynamics to network activity and its computational capabilities

Dendritic Slow Dynamics Enables Localized Cortical Activity to Switch between Mobile and Immobile Modes with Noisy Background Input

Mounting lines of evidence suggest the significant computational ability of a single neuron empowered by active dendritic dynamics. This motivates us to study what functionality can be acquired by a network of such neurons. The present paper studies how such rich single-neuron dendritic dynamics affects the network dynamics, a question which has scarcely been specifically studied to date. We simulate neurons with active dendrites networked locally like cortical pyramidal neurons, and find that naturally arising localized activity – called a bump – can be in two distinct modes, mobile or immobile. The mode can be switched back and forth by transient input to the cortical network. Interestingly, this functionality arises only if each neuron is equipped with the observed slow dendritic dynamics and with in vivo-like noisy background input. If the bump activity is considered to indicate a point of attention in the sensory areas or to indicate a representation of memory in the storage areas of the cortex, this would imply that the flexible mode switching would be of great potential use for the brain as an information processing device. We derive these conclusions using a natural extension of the conventional field model, which is defined by combining two distinct fields, one representing the somatic population and the other representing the dendritic population. With this tool, we analyze the spatial distribution of the degree of after-spike adaptation and explain how we can understand the presence of the two distinct modes and switching between the modes. We also discuss the possible functional impact of this mode-switching ability

Characterization of Voltage-Gated Ca2+ Conductances in Layer 5 Neocortical Pyramidal Neurons from Rats

Neuronal voltage-gated Ca2+ channels are involved in electrical signalling and in converting these signals into cytoplasmic calcium changes. One important function of voltage-gated Ca2+ channels is generating regenerative dendritic Ca2+ spikes. However, the Ca2+ dependent mechanisms used to create these spikes are only partially understood. To start investigating this mechanism, we set out to kinetically and pharmacologically identify the sub-types of somatic voltage-gated Ca2+ channels in pyramidal neurons from layer 5 of rat somatosensory cortex, using the nucleated configuration of the patch-clamp technique. The activation kinetics of the total Ba2+ current revealed conductance activation only at medium and high voltages suggesting that T-type calcium channels were not present in the patches. Steady-state inactivation protocols in combination with pharmacology revealed the expression of R-type channels. Furthermore, pharmacological experiments identified 5 voltage-gated Ca2+ channel sub-types – L-, N-, R- and P/Q-type. Finally, the activation of the Ca2+ conductances was examined using physiologically derived voltage-clamp protocols including a calcium spike protocol and a mock back-propagating action potential (mBPAP) protocol. These experiments enable us to suggest the possible contribution of the five Ca2+ channel sub-types to Ca2+ current flow during activation under physiological conditions

Rapid Mass Movement of Chloroplasts during Segment Formation of the Calcifying Siphonalean Green Alga, Halimeda macroloba

is abundant on coral reefs and is important in the production of calcium carbonate sediments. The process by which new green segments are formed over-night is revealed here for the first time. indicated that the movement process is dependent on both microtubules and microfilaments.This unusual process involves the mass movement of chloroplasts at a high rate into new segments during the night and rapid calcification on the following day and may be an adaptation to minimise the impact of herbivorous activity

Consequences of converting graded to action potentials upon neural information coding and energy efficiency

Information is encoded in neural circuits using both graded and action potentials, converting between them within single neurons and successive processing layers. This conversion is accompanied by information loss and a drop in energy efficiency. We investigate the biophysical causes of this loss of information and efficiency by comparing spiking neuron models, containing stochastic voltage-gated Na+ and K+ channels, with generator potential and graded potential models lacking voltage-gated Na+ channels. We identify three causes of information loss in the generator potential that are the by-product of action potential generation: (1) the voltage-gated Na+ channels necessary for action potential generation increase intrinsic noise and (2) introduce non-linearities, and (3) the finite duration of the action potential creates a ‘footprint’ in the generator potential that obscures incoming signals. These three processes reduce information rates by ~50% in generator potentials, to ~3 times that of spike trains. Both generator potentials and graded potentials consume almost an order of magnitude less energy per second than spike trains. Because of the lower information rates of generator potentials they are substantially less energy efficient than graded potentials. However, both are an order of magnitude more efficient than spike trains due to the higher energy costs and low information content of spikes, emphasizing that there is a two-fold cost of converting analogue to digital; information loss and cost inflation

Conjunctive input processing drives feature selectivity in hippocampal CA1 neurons

Feature-selective firing allows networks to produce representations of the external and internal environments. Despite its importance, the mechanisms generating neuronal feature selectivity are incompletely understood. In many cortical microcircuits the integration of two functionally distinct inputs occurs nonlinearly through generation of active dendritic signals that drive burst firing and robust plasticity. To examine the role of this processing in feature selectivity, we recorded CA1 pyramidal neuron membrane potential and local field potential in mice running on a linear treadmill. We found that dendritic plateau potentials were produced by an interaction between properly timed input from entorhinal cortex and hippocampal CA3. These conjunctive signals positively modulated the firing of previously established place fields and rapidly induced new place field formation to produce feature selectivity in CA1 that is a function of both entorhinal cortex and CA3 input. Such selectivity could allow mixed network level representations that support context-dependent spatial maps.Howard Hughes Medical InstituteRikagaku Kenkyūjo (Japan